A Hidden Semi-Markov Model-Based Speech Synthesis System

Recently, a statistical speech synthesis system based on the hidden Markov model (HMM) has been proposed. In this system, spectrum, excitation, and duration of human speech are modeled simultaneously by context-dependent HMMs and speech parameter vector sequences are generated from the HMMs themselves. This system defines a speech synthesis problem in a generative model framework and solves it using the maximum likelihood (ML) criterion. However, there is an inconsistency: although state duration models are explicitly used in the synthesis part of the system, they have not been incorporated in its training part. This inconsistency may degrade the naturalness of synthesized speech. In the present paper, a statistical speech synthesis system based on a hidden semi-Markov model (HSMM), which can be viewed as an HMM with explicit state duration models, is developed and evaluated. The use of HSMMs allows us to incorporate the state duration models explicitly not only in the synthesis part but also in the training part of the system and resolves the inconsistency in the HMM-based speech synthesis system. Subjective listening test results show that the use of HSMMs improves the reported naturalness of synthesized speech. key words: hidden Markov model, hidden semi-Markov model, HMMbased speech synthesis